Variational Cumulant Expansions for Intractable Distributions Pi Erre Van De Laar Rwcp (real World Computing Partnership) Theoretical Foundation Snn (foundation for Neural Networks)

Intractable distributions present a common diiculty in inference within the proba-bilistic knowledge representation framework and variational methods have recently been popular in providing an approximate solution. In this article, we describe a perturbational approach in the form of a cumulant expansion which, to lowest order, recovers the standard Kullback-Leibler variational bound. Higher-order terms describe corrections on the variational approach without incurring much further computational cost. The relationship to other perturbational approaches such as TAP is also elucidated. We demonstrate the method on a particular class of undirected graphical models, Boltzmann machines, for which our simulation results conrm improved accuracy and enhanced stability during learning.